Statistical models of images and early vision

A fundamental question in visual neuroscience is: Why are the receptive fields and response properties of visual neurons as they are? A modern approach to this problem emphasizes the importance of adaptation to ecologically valid input. In this paper, we will review work on modelling statistical regularities in ecologically valid visual input (“natural images”) and the obtained functional explanation of the properties of visual neurons. A seminal statistical model for natural images was linear sparse coding which is equivalent to the model called independent component analysis (ICA). Linear features estimated by ICA resemble wavelets or Gabor functions, and provide a very good description of the properties of simple cells in the primary visual cortex. We have introduced extensions of ICA that are based on modelling dependencies of the ”independent ” components estimated by basic ICA. The dependencies of the components are used to define either a grouping or a topographic order between the components. With natural image data, these models lead to emergence of further properties of visual neurons: the topographic organization and complex cell receptive fields. We have also modelled the temporal structure of natural image sequences, which provides an alternative approach to the sparseness used in most models. These models can be combined in a unifying framework that we call bubble coding. Finally, we will discuss a promising new direction of research: predictive visual neuroscience. There, the goal is to try to predict response properties of neurons in areas that are poorly understood, still based on statistical modelling of natural input. 1

Exploiting Sparse Representations for Robust Analysis of Noisy Complex Video Scenes

Abstract. Recent works have shown that, even with simple low level visual cues, complex behaviors can be extracted automatically from crowded scenes, e.g. those depicting public spaces recorded from video surveillance cameras. However, low level features as optical flow or fore-ground pixels are inherently noisy. In this paper we propose a novel unsupervised learning approach for the analysis of complex scenes which is specifically tailored to cope directly with features ’ noise and uncer-tainty. We formalize the task of extracting activity patterns as a matrix factorization problem, considering as reconstruction function the robust Earth Mover’s Distance. A constraint of sparsity on the computed basis matrix is imposed, filtering out noise and leading to the identification of the most relevant elementary activities in a typical high level behavior. We further derive an alternate optimization approach to solve the pro-posed problem efficiently and we show that it is reduced to a sequence of linear programs. Finally, we propose to use short trajectory snippets to account for object motion information, in alternative to the noisy optical flow vectors used in previous works. Experimental results demonstrate that our method yields similar or superior performance to state-of-the arts approaches.

Instant action recognition

Abstract. In this paper, we present an efficient system for action recognition from very short sequences. For action recognition typically appearance and/or motion information of an action is analyzed using a large number of frames. This is a limitation if very fast actions (e.g., in sport analysis) have to be analyzed. To overcome this limitation, we propose a method that uses a single-frame representation for actions based on appearance and motion information. In particular, we estimate Histograms of Oriented Gradients (HOGs) for the current frame as well as for the corresponding dense flow field. The thus obtained descriptors are efficiently represented by the coefficients of a Non-negative Matrix Factorization (NMF). Actions are classified using an one-vs-all Support Vector Machine. Since the flow can be estimated from two frames, in the evaluation stage only two consecutive frames are required for the action analysis. Both, the optical flow as well as the HOGs, can be computed very efficiently. In the experiments, we compare the proposed approach to state-of-the-art methods and show that it yields competitive results. In addition, we demonstrate action recognition for real-world beach-volleyball sequences.

Detecting Regional Abnormal Cardiac Contraction in Short-Axis MR Images Using Independent Component Analysis

Abstract. Regional myocardial motion analysis is used in clinical rou-tine to inspect cardiac contraction in myocardial diseases such as in-farction or hypertrophy. Physicians/radiologists can recognize abnormal cardiac motion because they have knowledge about normal heart con-traction. This paper explores the potential of Independent Component Analysis (ICA) to extract local myocardial contractility patterns and to use them for the automatic detection of regional abnormalities. A quali-tative evaluation was performed using 42 healthy volunteers to train the ICA model and 6 infarct patients to test the detection and localization. This experiment shows that the evaluation results correlate very well to the clinical gold standard: delayed-enhancement MR images.